One-dimensional convolutions

In this short post we build on our previous discussion of single-dimensional moving averages by describing their generalization as single-dimensional convolutions. As in that prior post here we too will be working with a generic ordered sequence of $P$ points denoted as

\begin{equation} x_1,\, x_2,\, \ldots,\, x_P. \end{equation}

You can skip around this document to particular subsections via the hyperlinks below.

InΒ [Β ]:

Generalizing the moving averageΒΆ

In our previous post on moving averages) we saw the moving average of order or window-length $D$ is itself a sequence of the same length, with its first $D$ values equal to the sequence itself as

\begin{equation} h_p = x_p \,\,\,\,\,\, p=1,...,D. \end{equation}

and in general for $p \geq D+1$ we have

\begin{equation} h_{p} = \frac{x_{p-1} + x_{p-2} + \cdots + x_{p-D}}{D}. \end{equation}

The functionality used here (the simple average) can in principle be replaced with any general function $f$ one desires as

\begin{equation} h_{p} = f\left(x_{p-1},x_{p-2},\cdots,x_{p-D}\right). \end{equation}

However if we swap out the average with something slightly more complicated (relatively speaking) - the linear combination - we would instead compute updates of the form

\begin{equation} h_{p} = w_1x_{p-1} + w_2x_{p-2} + \cdots + w_{D}x_{p-D}. \end{equation}

Here the weights $w_1,\,w_2,\,...,w_{D}$ can be set however we please, and if set to $w_d = \frac{1}{D}$ for all $d$ we recover the moving average update above.

The set of such linear combinations are often referred to as convolutions, and the weights $w_1,\,...,w_{D}$ as filters or kernels (see endnotes for further discussion of this term).

Like the moving average, convolutions dynamic systems that are often used to smooth and - more generally - clean up time series for further processing. As with the moving average, general convolutions

Common convolution filtersΒΆ

InΒ [2]:

Viewed through the lens of convolution, a moving average is often referred to as a "mean filter". Below we produce two Python functions that illustrate this perspective, and allow for immediate generalization to other filters. The mean_weights function creates a set of "mean weights" used to produce a moving average - that is, the elements of a series in each window of length $D$ are multiplied by these weights. This function takes in only one input: $D$ the order of the moving average.

InΒ [3]:
# make average weights
def mean_weights(D):
    weights = [1/D for v in range(D)]
    return weights

We can visualize these weights - as shown below for the case where $\mathcal{D} = 10$. Here each black dot represents a weight value, and the dashed lines are drawn for visualization purposes only.

InΒ [8]:

We can then produce a moving average via the linear_filter function below - which takes in a time series x, a window length O, and a set of filter weights w.

InΒ [9]:
# general linear filter function
def linear_filter(x,w):
    # filter input signal
    D = len(w)
    y = [v for v in x[:D]]
    for p in range(len(x) - D):
        # make next element
        b = sum([a*b for a,b in zip(x[p:p+D],w)])
        y.append(b)
    return np.array(y)

This process, animated below for a particular time series, smooths the input time series. As you move the slider from left to right you will see the window in which each average is computed, straddled on both sides by vertical blue bars, move from left to right across the series with the resulting moving average shown as a pink series.

InΒ [10]:
Out[10]:



Other popular choices for weights outside of the average include average cosine and sinc weights. These arise naturally in the study of signal processing where they go by names like "low pass filtering". To create such weights we sample an elementary function, like for example the sinc function shown below for $D=30$, and multiply the resulting weights by windowed portions of our input series precisely as we did with the original moving average weights above.

InΒ [11]:
InΒ [12]:

To produce this set of sinc weights we simply adjust the basic sinc function, and divide off the order of the window as shown below.

InΒ [13]:
# compute centered sinc weights
def sinc_weights(D,**kwargs):
    v = np.arange(0.5-D/2,D/2,1)
    freq = 1
    if 'freq' in kwargs:
        freq = kwargs['freq']
    weights = []
    for o in v:
        spot = o/D
        if np.abs(spot) < 10**(-8):
            spot = 1
        h = np.sin(2*np.pi*freq*spot)/(2*np.pi*freq*spot)
        weights.append(h)
    
    # normalize
    h_sum = sum(weights)
    weights = [h/h_sum for h in weights]
    return weights

This set of weights will smooth an input series, leaving a more trigonometric fit than the moving average.

InΒ [14]:
Out[14]:



By similarly sampling a cosine we can create cosine filter - whose weights are shown below.

InΒ [15]:

EndnotesΒΆ

The general linear combination in Equation (4) is more accurately known as cross-correlation, which is closely related to the convolution operation which is more commonly defined as

\begin{equation} h_{p} = w_{D}x_{p-1} + w_{D-1}x_{p-2} + \cdots + w_{1}x_{p-D}. \end{equation}

Here the weight sequence $w$ has been reversed before being multiplied by the elements of $x$. Flipping the order of the weights makes for a more awkward formula, and gaurntees certain mathematical niceties like e.g., the commutative property. But because these mathematical ideas do not concern us - and for the sake of clarity in notation - we mish-mash the terms corss-correlation and convolution.